1. Field of the Invention
This invention relates to neural networks and, more particularly, to a method of storing associative information in neural networks and a device therefor.
2. Description of the Prior Art
An associative information storage algorithm in the neural networks can be described by utilizing state transition dynamics of the neural networks with an associative matrix "W" and state vector "X" of the neural networks, such as disclosed in, for example, J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities" Proceedings of the National Academy of Sciences 79, pp. 2554-2558.
The state transition dynamics are normally defined by EQU x(t+1)=f{Wx(t)} (1)
where, x(t) indicates a state of the neural networks at a time t.
A typical conventional associative information storage system is shown in FIG. 1, and FIG. 2 illustrates a nonlinear function of state change of the associative information storage system of FIG. 1.
Now referring to FIG. 1, block 30 designates interconnection type neural networks for storing associative matrices, block 4 designates a state input connecting unit, block 5 designates a state output connecting unit and block 6 designates a conversion unit of the type for converting data in accordance with the nonlinear transform function "f" defined by equation (1) above, whereby an associative information storage circuit 20 is formed of the neural networks 30, the state input connecting unit 4, the state output connecting unit 5 and the conversion unit 6. Further, block 7 designates an external device coupled to the associative information storage circuit 20 for storing a state of the associative information storage circuit 20, and block 80 designates an initial value input unit to provide the associative information storage circuit 20 with an initial state.
Heretofore, the associative matrix has been provided by a sum of autocorrelation matrices of storage patterns gained through a nonlinear transform function, such as a step function shown in FIG. 2, a sigmoid function shown in FIG. 3 and the like.
The essential operation of the associative information storage device will now be described hereinafter.
In accordance with the associative matrices as described above, since a storage pattern takes part of an invariant of equation (1), the storage pattern may be stored in the associative matrices as the invariant of the state transition dynamics. In addition to this, the storage pattern can be associated from the input pattern through a convergence point by providing the state with a transition in accordance with equation (1) on the condition that the input pattern is used for the initial value of the neural networks and by making use of the characteristic such a state of the neural networks will coincide with the storage pattern when a sufficient time has elapsed after the commencement of a state transition from an initial state provided that the initial state is little different from the storage pattern of equation (1).
The neural networks 30 which constitute part of the associative information storage circuit 20 as described above may be realized by utilizing an integrated circuit in accordance with an optical and electronic method. The realization of such neural networks will be described in more detail hereinafter.
An optical filter having a distribution of light transmittance, which is equivalent to the associative matrices, is formed on a two-dimensional plane indicated by the neural networks 30 of FIG. 1, and then the formed optical filter is interposed between a light source, which constitutes an input pattern, and a photo-detector to compute a result.
The operation of the neural networks 30 will be described hereinafter. The light emitted from the light source corresponding to the input pattern passes through the optical filter, which has a distribution of light transmittance that corresponds to a supplementary matrix. Since each element of the optical filter has a light transmittance that corresponds to a coupling power between the light source and the photo-detector, the strength of the light which has passed through the optical filter is equal to a product of the light intensity of the light source and the light transmittance of each optical filter element, whereby the light transmittance corresponds to the respective coupling power. The photo-detector outputs a signal by adding the light intensity of the light emitted from all of the elements of the light source that passed through the optical filter. Accordingly, it is possible to find a product of the associative matrices and a vector indicating the state input by use of the optical system which constitutes the neural networks 30.
Since the prior art associative information storage device has a configuration as described above, there have been problems in that it requires neural networks having associative matrices which are of a size equal to the size of input pattern squared. It is also necessary to convert a two-dimensional pattern, such as an image, once into a one-dimensional pattern, and a capacity of the associated information storage is limited to a small amount, on such an order of 0.15 n when the size of the state of the neural networks is designated by "n".